Header minimieren

Prof. Dr.-Ing. Marcus Barkowsky

Forschung:

Quality of Experience
Algorithmen für Videoqualitätsanalyse
Modellierung des menschlichen visuellen Systems
Durchführung und Auswertung perzeptiver Vergleichsstudien
Perzeptive Verfahren in der Videocodierung

Lehre:

Interaktion
Internet of Things, Netzwerke
Mikrorechnertechnik
Bild- und Videoverarbeitung, inkl. tiefer neuronaler Netze
Digitale Signalverarbeitung
Informatik, insb. Software Engineering

Fakultät Angewandte Informatik

Professorinnen und Professoren

Prodekan

Prodekan der Fakultät “Angewandte Informatik” Studiengangskoordinator “Interaktive Systeme und Internet of Things”

DEGG's 2.17

0991/3615-409

Sprechzeiten

Freitag, 12:45-13:30 nach Email-Vereinbarung

Publikationen

Sortierung:

Beitrag in Sammelwerk/Tagungsband

J. Bialkowski
M. Menden
Marcus Barkowsky
A. Kaup

A Fast H.263 to H.264 Inter-Frame Transcoder with Motion Vector Refinement.

San Francisco pg. 47-52

(2004)

Beitrag in Sammelwerk/Tagungsband

J. Bialkowski
Marcus Barkowsky
A. Kaup

On Requantization in Intra-Frame Video Transcoding with Different Transform Block Sizes.

pg. 1-4

DOI: 10.1109/MMSP.2005.248669

(2005)

Beitrag in Sammelwerk/Tagungsband

Marcus Barkowsky
J. Bialkowski
A. Kaup

Subjektiver Videobetrachtungstest für niederratige Multimedia-Szenarien.

VDE-Verlag pg. 169-175

(2005)

Beitrag in Sammelwerk/Tagungsband

J. Bialkowski
Marcus Barkowsky
F. Leschka
A. Kaup

Low-Complexity Transcoding of Inter Coded Video Frames from H.264 to H.263.

pg. 837-840

(2006)

Beitrag in Sammelwerk/Tagungsband

J. Bialkowski
Marcus Barkowsky
A. Kaup

A new algorithm for reducing the requantization loss in video transcoding.

pg. 1-5

(2006)

Beitrag in Sammelwerk/Tagungsband

U. Fecker
Marcus Barkowsky
A. Kaup

Improving the Prediction Efficiency for Multi-View Video Coding Using Histogram Matching.

In:
Y. He

[Isernhagen]: [Kenzler Conference Management]

(2006)

Beitrag in Sammelwerk/Tagungsband

Marcus Barkowsky
B. Eskofier
J. Bialkowski
A. Kaup

Influence of the Presentation Time on Subjective Votings of Coded Still Images.

IEEE pg. 429-432

(2006)

Beitrag in Sammelwerk/Tagungsband

J. Bialkowski
Marcus Barkowsky
A. Kaup

Overview of Low-Complexity Video Transcoding from H.263 to H.264.

pg. 49-52

(2006)

Beitrag in Sammelwerk/Tagungsband

Marcus Barkowsky
R. Bitto
J. Bialkowski
A. Kaup
B. Li

Comparison of matching strategies for temporal frame registration in the perceptual evaluation of video quality.

(2006)

Beitrag in Sammelwerk/Tagungsband

Marcus Barkowsky
J. Bialkowski
R. Bitto
A. Kaup

Temporal Registration using 3D Phase Correlation and a Maximum Likelihood Approach in the Perceptual Evaluation of Video Quality.

pg. 195-198

(2007)

Beitrag in Sammelwerk/Tagungsband

Marcus Barkowsky
B. Eskofier
R. Bitto
J. Bialkowski
A. Kaup

Perceptually motivated spatial and temporal integration of pixel based video quality measures.

Association for Computer Machinery, ACM pg. 1-7

(2007)

Beitrag in Sammelwerk/Tagungsband

U. Fecker
Marcus Barkowsky
A. Kaup

Time-Constant Histogram Matching for Luminance and Chrominance Compensation of Multi-View Video Sequences.

Lisbon, Portugal

(2007)

Zeitschriftenartikel

J. Bialkowski
Marcus Barkowsky
A. Kaup

Fast Video Transcoding from H.263 To H. 264/AVC.

In: Multimedia Tools and Applications (vol. 35) , pg. 127-146

(2007)

DOI: 10.1007/s11042-007-0126-7

Abstract anzeigen

In the past 10 years detailed works on different video transcoders have been published. However, the new ITU-T Recommendation H.264—also adapted as ISO/IEC MPEG-4 Part 10 (AVC)—provides many new encoding options for the prediction processes that lead to difficulties for low complexity transcoding. In this work we present very fast transcoding techniques to convert H.263 bitstreams into H.264/AVC bitstreams. We will give reasoning, why the proposed pixel domain approach is advantageous in this scenario instead of using a DCT domain transcoder. Our approach results in less than 9% higher data rate at equivalent PSNR quality compared to a full-search approach. But this rate loss allows the reduction of the search complexity by a factor of over 200 for inter frames and still a reduction of over 70% for intra frames. A comparison to a fast search algorithm is given. We also provide simulation results that our algorithm works for transcoding MPEG-2 to H.264/AVC in the aimed scenario.

Zeitschriftenartikel

U. Fecker
Marcus Barkowsky
A. Kaup

Histogram-Based Prefiltering for Luminance and Chrominance Compensation of Multiview Video.

In: IEEE Transactions on Circuits and Systems for Video Technology (vol. 18) , pg. 1258-1267

(2008)

Abstract anzeigen

Significant advances have recently been made in the coding of video data recorded with multiple cameras. However, luminance and chrominance variations between the camera views may deteriorate the performance of multiview codecs and image-based rendering algorithms. A histogram matching algorithm can be applied to efficiently compensate for these differences in a prefiltering step. A mapping function is derived which adapts the cumulative histogram of a distorted sequence to the cumulative histogram of a reference sequence. If all camera views of a multiview sequence are adapted to a common reference using histogram matching, the spatial prediction across camera views is improved. The basic algorithm is extended in three ways: a time-constant calculation of the mapping function, RGB color conversion, and the use of global disparity compensation. The best coding results are achieved when time-constant histogram calculation and RGB color conversion are combined. In this case, the usage of histogram matching prior to multiview encoding leads to substantial gains in the coding efficiency of up to 0.7 dB for the luminance component and up to 1.9 dB for the chrominance components. This prefiltering step can be combined with block-based illumination compensation techniques that modify the coder and decoder themselves, especially with the approach implemented in the multiview reference software of the joint video team (JVT). Additional coding gains up to 0.4 dB can be observed when both methods are combined.

Zeitschriftenartikel

Marcus Barkowsky
J. Bialkowski
B. Eskofier
R. Bitto
A. Kaup

Temporal Trajectory Aware Video Quality Measure.

In: IEEE Journal of Selected Topics in Signal Processing (vol. 3) , pg. 266-279

(2009)

Abstract anzeigen

The measurement of video quality for lossy and low-bitrate network transmissions is a challenging topic. Especially, the temporal artifacts which are introduced by video transmission systems and their effects on the viewer's satisfaction have to be addressed. This paper focuses on a framework that adds a temporal distortion awareness to typical video quality measurement algorithms. A motion estimation is used to track image areas over time. Based on the motion vectors and the motion prediction error, the appearance of new image areas and the display time of objects is evaluated. Additionally, degradations which stick to moving objects can be judged more exactly. An implementation of this framework for multimedia sequences, e.g., QCIF, CIF, or VGA resolution, is presented in detail. It shows that the processing steps and the signal representations that are generated by the algorithm follow the reasoning of a human observer in a subjective experiment. The improvements that can be achieved with the newly proposed algorithm are demonstrated using the results of the Multimedia Phase I database of the Video Quality Experts Group.

Beitrag in Sammelwerk/Tagungsband

Marcus Barkowsky
R. Cousseau
P. Le Callet

Influence of Depth Rendering on the Quality of Experience for an Autostereoscopic Display.

(2009)

Hochschulschrift

Marcus Barkowsky

Subjective and Objective Video Quality Measurement in Low-Bitrate Multimedia Scenarios.

Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen.

(2009)

Beitrag in Sammelwerk/Tagungsband

Y. Pitrey
Marcus Barkowsky
P. Le Callet
R. Pépion

Subjective Quality Assessment Of MPEG-4 Scalable Video Coding In a Mobile Scenario.

Paris, France pg. 72

(2010)

Beitrag in Sammelwerk/Tagungsband

Marcus Barkowsky
P. Le Callet

The influence of autostereoscopic 3D displays on subsequent task performance.

In:
SPIE - The International Society for Optical Engineering

pg. 7524-7534

(2010)

Beitrag in Sammelwerk/Tagungsband

W. Chen
J. Fournier
Marcus Barkowsky
P. Le Callet

New Requirements of Subjective Video Quality Assessment Methodologies for 3DTV.

(2010)

Beitrag in Sammelwerk/Tagungsband

Marcus Barkowsky
K. Wang
R. Cousseau
K. Brunnström
R. Olsson
P. Le Callet

Subjective Quality Assessment of Error Concealment Strategies for 3DTV in the presence of asymmetric Transmission Errors.

(2010)

Beitrag in Sammelwerk/Tagungsband

Marcus Barkowsky
P. Campisi
P. Le Callet
V. Rizzo

Crosstalk measurement and mitigation for autostereoscopic displays.

In:
SPIE - The International Society for Optical Engineering

pg. 7526-7531

(2010)

Beitrag in Sammelwerk/Tagungsband

U. Engelke
Marcus Barkowsky
P. Le Callet
H.-J. Zepernick

Modelling Saliency Awareness for Objective Video Quality Assessment.

(2010)

Beitrag in Sammelwerk/Tagungsband

Marcus Barkowsky
P. Le Callet

On the Perceptual Similarity of Realistic Looking Tone Mapped High Dynamic Range Images.

(2010)

Beitrag in Sammelwerk/Tagungsband

Q. Huynh-Thu
Marcus Barkowsky
P. Le Callet

Video Quality Assessment: From 2D to 3D ‐ Challenges and Future Trends.

(2010)

Beitrag in Sammelwerk/Tagungsband

U. Engelke
Marcus Barkowsky
P. Le Callet
H.-J. Zepernick

Modelling Saliency Awareness for Objective Video Quality Assessment.

Trondheim, Norway

(2010)

Beitrag in Sammelwerk/Tagungsband

Y. Pitrey
Marcus Barkowsky
P. Le Callet
R. Pépion

Subjective Quality Evaluation of H.264 High-Definition Video Coding versus Spatial Up-Scaling and Interlacing.

Tampere, Finlande

(2010)

Beitrag in Sammelwerk/Tagungsband

Marcus Barkowsky
M. Pinson
R. Pépion
P. Le Callet

Analysis of Freely Available Dataset for HDTV including Coding and Transmission Distortions.

(2010)

Beitrag in Sammelwerk/Tagungsband

Y. Pitrey
Marcus Barkowsky
P. Le Callet
R. Pépion

Evaluation of MPEG4-SVC for QoE Protection in the Context of Transmission Errors.

In:
SPIE - The International Society for Optical Engineering

San Diego, CA, USA

(2010)

Beitrag in Sammelwerk/Tagungsband

J. Li
Marcus Barkowsky
Wang, J.
P. Le Callet

Study on Visual Discomfort Induced by Stimulus Movement at Fixed Depth on Stereoscopic Displays using Shutter Glasses.

(2011)

Beitrag in Sammelwerk/Tagungsband

Y. Pitrey
U. Engelke
P. Le Callet
Marcus Barkowsky
R. Pépion

Subjective Quality of SVC-coded Videos with different Error-Patterns concealed using Spatial Scalability.

Paris, France pg. 67

(2011)

Beitrag in Sammelwerk/Tagungsband

W. Chen
J. Fournier
Marcus Barkowsky
P. Le Callet

New stereoscopic video shooting rule based on stereoscopic distortion parameters and comfortable viewing zone.

In:
SPIE - The International Society for Optical Engineering

pg. 78631O

(2011)

Zeitschriftenartikel

Q. Huynh-Thu
Marcus Barkowsky
P. Le Callet

The Importance of Visual Attention in Improving the 3D-TV Viewing Experience: Overview and New Perspectives.

In: IEEE Transactions on Broadcasting (vol. 57) , pg. 421-431

(2011)

Abstract anzeigen

Three-dimensional video content has attracted much attention in both the cinema and television industries, because 3D is considered to be the next key feature that can significantly enhance the visual experience of viewers. However, one of the major challenges is the difficulty in providing high quality images that are comfortable to view and that also meet signal transmission requirements over a limited bandwidth for display on television screens. The different processing steps that are necessary in a 3D-TV delivery chain can all introduce artifacts that may create problems in terms of human visual perception. In this paper, we highlight the importance of considering 3D visual attention when addressing 3D human factors issues. We provide a review of the field of 3D visual attention, discuss the challenges in both the understanding and modeling of 3D visual attention, and provide guidance to researchers in this field. Finally, we identify perceptual issues generated during the various steps in a typical 3D-TV broadcasting delivery chain, review them and explain how consideration of 3D visual attention modeling can help improve the overall 3D viewing experience.

Beitrag in Sammelwerk/Tagungsband

Marcus Barkowsky
R. Cousseau
P. Le Callet

Is visual fatigue changing the perceived depth accuracy on an autostereoscopic display?.

In:
SPIE - The International Society for Optical Engineering

pg. 78631V

(2011)

Beitrag in Sammelwerk/Tagungsband

N. Staelens
I. Sedano
Marcus Barkowsky
L. Janowski
K. Brunnström
P. Le Callet

Standardized Toolchain And Model Development For Video Quality Assessment ‐ The Mission Of The Joint Effort Group In Vqeg.

Mechelen, Belgique pg. 61

(2011)

Beitrag in Sammelwerk/Tagungsband

K. Wang
Marcus Barkowsky
R. Cousseau
K. Brunnström
R. Olsson
P. Le Callet
M. Sjöström

Subjective evaluation of HDTV stereoscopic videos in IPTV scenarios using absolute category rating.

In:
SPIE - The International Society for Optical Engineering

pg. 78631T

(2011)

Beitrag in Sammelwerk/Tagungsband

Y. Pitrey
U. Engelke
Marcus Barkowsky
R. Pépion
P. Le Callet

Aligning Subjective Tests using a Low Cost Common Set.

Lisbon, Portugal

(2011)

Beitrag in Sammelwerk/Tagungsband

P. Lebreton
A. Raake
Marcus Barkowsky
P. Le Callet

A Subjective Evaluation of 3D IPTV Broadcasting Implementations Considering Coding and Transmission Degradation.

pg. 506-511

(2011)

Beitrag in Sammelwerk/Tagungsband

Wang, J.
Marcus Barkowsky
V. Ricordel
P. Le Callet

Quantifying how the combination of blur and disparity affects the perceived depth.

In:
SPIE - The International Society for Optical Engineering

pg. 78650K

(2011)

Beitrag in Sammelwerk/Tagungsband

J. Li
Marcus Barkowsky
P. Le Callet

Visual Discomfort Induced by Relative Disparity and Planar Motion of Stereoscopic Images.

Nantes, France pg. 1-2

(2011)

Beitrag in Sammelwerk/Tagungsband

Wang, J.
Marcus Barkowsky
V. Ricordel
P. Le Callet

Quantifying how the Combination of Blur and Disparity affects the Perceived Depth.

In:
SPIE - The International Society for Optical Engineering

pg. 78650K

(2011)

Zeitschriftenartikel

Marcus Barkowsky
S. Tourancheau
K. Brunnström
K. Wang
B. Andrén

55.3: Crosstalk Measurements of Shutter Glasses 3D Displays.

In: SID Symposium Digest of Technical Papers (vol. 42) , pg. 812-815

(2011)

Abstract anzeigen

Crosstalk is probably one of the main perceptual factors contributing to perceived image quality and visual comfort. The Video Quality Experts Group (VQEG) within their 3D video quality project is specifying a practical measurement procedure that will produce consistent results across laboratories. This paper is part of that effort. Two different method of measuring cross talk on shutter glasses stereo displays have been studied. One is based on time average luminance measurements and the other on temporal measurements. The results show that crosstalk is roughly 0.5% but that there are differences in the crosstalk between the two eyes in the shutter glasses.

Beitrag in Sammelwerk/Tagungsband

J. Li
Marcus Barkowsky
P. Le Callet

The Influence of Relative Disparity and Planar Motion Velocity on Visual Discomfort of Stereoscopic Videos.

Mechelen, Belgique

(2011)

Beitrag in Sammelwerk/Tagungsband

Wang, J.
Marcus Barkowsky
V. Ricordel
P. Le Callet

Clarifying how defocus blur and disparity affect the perceived depth.

Nantes, France pg. 1

(2011)

Beitrag in Sammelwerk/Tagungsband

M. Urvoy
Marcus Barkowsky
J. Gutiérrez
R. Cousseau
Y. Koudota
V. Ricordel
P. Le Callet
N. García

NAMA3DS1-COSPAD1: Subjective video quality assessment database on coding conditions introducing freely available high quality 3D stereoscopic sequences.

(2012)

Beitrag in Sammelwerk/Tagungsband

W. Chen
J. Fournier
Marcus Barkowsky
P. Le Callet

Quality of experience model for 3DTV.

In:
G. Favalora
N. Holliman
A. Woods

San Francisco, CA, USA vol. 8288 pg. 1-6

DOI: 10.1117/12.907873

(2012)

Beitrag in Sammelwerk/Tagungsband

Marcus Barkowsky
N. Staelens
L. Janowski
Y. Koudota
M. Leszczuk
M. Urvoy
P. Hummelbrunner
I. Sedano
K. Brunnström

Subjective experiment dataset for joint development of hybrid video quality measurement algorithms.

Berlin pg. 1-4

(2012)

Beitrag in Sammelwerk/Tagungsband

S. Tourancheau
K. Wang
J. Bulat
R. Cousseau
L. Janowski
K. Brunnström
Marcus Barkowsky

Reproducibility of crosstalk measurements on active glasses 3D LCD displays based on temporal characterization.

In:
SPIE - The International Society for Optical Engineering

(2012)

Beitrag in Sammelwerk/Tagungsband

J. Li
Marcus Barkowsky
P. Le Callet

Analysis and Improvement of a Paired Comparison Method in the Application of 3DTV Subjective Experiment.

(2012)

Beitrag in Sammelwerk/Tagungsband

Y. Pitrey
R. Pépion
P. Le Callet
Marcus Barkowsky

Using overlapping subjective datasets to assess the performance of objective quality metrics on Scalable Video Coding and error concealment.

Melbourne, Australia pg. 103

(2012)

Beitrag in Sammelwerk/Tagungsband

P. Lebreton
A. Raake
Marcus Barkowsky
P. Le Callet

Perceptual depth indicator for S-3D content based on binocular and monocular cues.

pg. 734-738

(2012)

Beitrag in Sammelwerk/Tagungsband

P. Lebreton
A. Raake
U. Wustenhagen
T. Buchholz
Marcus Barkowsky
P. Le Callet

A subjective and objective evaluation of a realistic 3D IPTV transmission chain.

München pg. 179

(2012)

Beitrag in Sammelwerk/Tagungsband

Marcus Barkowsky
M. Blestel
M. Carnec
A. Ksentini
P. Le Callet
G. Madec
R. Monnier
J. Nezan
R. Pepion
Y. Pitrey
J-F Travers
M. Raulet
A. Untersee

An Overview of the SVC4QoE project.

In:
J. Rodriguez
R. Tafazolli
C. Verikoukis

Springer pg. 560-570

(2012)

Zeitschriftenartikel

K. Wang
Marcus Barkowsky
K. Brunnström
M. Sjöström
R. Cousseau
P. Le Callet

Perceived 3D TV Transmission Quality Assessment: Multi-Laboratory Results Using Absolute Category Rating on Quality of Experience Scale.

In: IEEE Transactions on Broadcasting (vol. 58) , pg. 544-557

(2012)

Abstract anzeigen

Inspired by the rapidly increasing popularity of 3D movies, there is an industrial push for 3DTV services to the home. One important factor for the success and acceptance by the viewers is a positive quality of experience (QoE) of the new service when delivered. The questions of how to efficiently deliver 3DTV service to the home, and how to evaluate the visual quality perceived by end users are a recent research focus. We have investigated users\textquoteright experience of stereoscopic 3D video quality by preparing two subjective assessment datasets. The first dataset aimed at the evaluation of efficient transmission in the transmission error free case, while the second focused on error concealment. A total of three subjective assessments, two for first dataset and one for the second, were performed using the Absolute Category Rating with Hidden unimpaired Reference video (ACR-HR) method. The experimental setup allows to show that the ACR-HR subjective method provides repeatable results across labs and across conditions for video quality. It was also verified that MVC is more efficient than H.264 simulcast coding. Furthermore it was discovered that based on the same level of quality of experience spatial down-sampling may lead to better bitrate efficiency while temporal down-sampling is not acceptable. When network impairments occur, traditional error 2D concealment methods need to be reinvestigated as they were outperformed by displaying the same view for both eyes (switching to 2D presentation).

Zeitschriftenartikel

P. Lebreton
A. Raake
Marcus Barkowsky
P. Le Callet

Evaluating Depth Perception of 3D Stereoscopic Videos.

In: IEEE Journal of Selected Topics in Signal Processing (vol. 6) , pg. 710-720

(2012)

Abstract anzeigen

3D video quality of experience (QoE) is a multidimensional problem; many factors contribute to the global rating like image quality, depth perception and visual discomfort. Due to this multidimensionality, it is proposed in this paper, that as a complement to assessing the quality degradation due to coding or transmission, the appropriateness of the non-distorted signal should be addressed. One important factor here is the depth information provided by the source sequences. From an application-perspective, the depth-characteristics of source content are of relevance for pre-validating whether the content is suitable for 3D video services. In addition, assessing the interplay between binocular and monocular depth features and depth perception are relevant topics for 3D video perception research. To achieve the evaluation of the suitability of 3D content, this paper describes both a subjective experiment and a new objective indicator to evaluate depth as one of the added values of 3D video.

Beitrag in Sammelwerk/Tagungsband

Y. Pitrey
P. Hummelbrunner
B. Kitzinger
S. Buchinger
Marcus Barkowsky
P. Le Callet
R. Pepion

Influence of Shooting Conditions, Re-Encoding and Viewing Conditions on the Perceived Quality of User-Generated Videos.

(2012)

Beitrag in Sammelwerk/Tagungsband

K. Brunnström
I. Sedano
K. Wang
Marcus Barkowsky
M. Kihl
P. Le Callet
Patrick
M. Sjöström
A. Aurelius

2D No-Reference Video Quality Model Development and 3D Video Transmission Quality.

(2012)

Beitrag in Sammelwerk/Tagungsband

W. Chen
J. Fournier
Marcus Barkowsky
P. Le Callet

Exploration of Quality of Experience of Stereoscopic Images: Binocular Depth.

(2012)

Zeitschriftenartikel

M. Pinson
L. Janowski
R. Pepion
Q. Huynh-Thu
C. Schmidmer
P. Corriveau
A. Younkin
P. Le Callet
Marcus Barkowsky
W. Ingram

The Influence of Subjects and Environment on Audiovisual Subjective Tests: An International Study.

In: IEEE Journal of Selected Topics in Signal Processing (vol. 6) , pg. 640-651

(2012)

DOI: 10.1109/JSTSP.2012.2215306

Abstract anzeigen

Traditionally, audio quality and video quality are evaluated separately in subjective tests. Best practices within the quality assessment community were developed before many modern mobile audiovisual devices and services came into use, such as internet video, smart phones, tablets and connected televisions. These devices and services raise unique questions that require jointly evaluating both the audio and the video within a subjective test. However, audiovisual subjective testing is a relatively under-explored field. In this paper, we address the question of determining the most suitable way to conduct audiovisual subjective testing on a wide range of audiovisual quality. Six laboratories from four countries conducted a systematic study of audiovisual subjective testing. The stimuli and scale were held constant across experiments and labs; only the environment of the subjective test was varied. Some subjective tests were conducted in controlled environments and some in public environments (a cafeteria, patio or hallway). The audiovisual stimuli spanned a wide range of quality. Results show that these audiovisual subjective tests were highly repeatable from one laboratory and environment to the next. The number of subjects was the most important factor. Based on this experiment, 24 or more subjects are recommended for Absolute Category Rating (ACR) tests. In public environments, 35 subjects were required to obtain the same Student\textquoterights t-test sensitivity. The second most important variable was individual differences between subjects. Other environmental factors had minimal impact, such as language, country, lighting, background noise, wall color, and monitor calibration. Analyses indicate that Mean Opinion Scores (MOS) are relative rather than absolute. Our analyses show that the results of experiments done in pristine, laboratory environments are highly representative of those devices in actual use, in a typical user environment.

Beitrag in Sammelwerk/Tagungsband

Y. Pitrey
Marcus Barkowsky
R. Pépion
P. Le Callet
H. Hlavacs

Influence of the source content and encoding configuration on the preceived quality for scalable video coding.

In:
SPIE - The International Society for Optical Engineering

San Francisco, CA, USA pg. 1-6

(2012)

Beitrag in Sammelwerk/Tagungsband

K. Wang
K. Brunnström
Marcus Barkowsky
M. Urvoy
M. Sjöström
P. Le Callet
S. Tourancheau
B. Andrén

Stereoscopic 3D video coding quality evaluation with 2D objective metrics.

In:
SPIE - The International Society for Optical Engineering

(2013)

Zeitschriftenartikel

J. Li
Marcus Barkowsky
P. Le Callet

Recent Advances in Standardization on 3D Quality of Experience.

In: IEEE COMSOCMMTC E-Letter (vol. 8) , pg. 20

(2013)

Abstract anzeigen

For the last decades, video quality assessment has mostly tackled 2D video sequences. Technological advances were mostly tackling coding and transmission schemes while the display technology, especially in lab viewing environments, could be considered as transparent. Subjective assessment methodologies needed to be selected mostly with respect to the severity of the degradations. Typical examples are Absolute Category Rating with Hidden Reference (ACR-HR) from ITU-T P.910 for strong degradations experienced in networked multimedia scenarios, and Paired Comparison (PC) or Double Stimulus Continuous Quality Scale (DSCQS) from ITU-R BT.500 for near lossless scenarios such as satellite transmissions.

Beitrag in Sammelwerk/Tagungsband

J. Li
Marcus Barkowsky
P. Le Callet

Visual Discomfort is not always proportonal to Eye Blinking Rate: Exploring Some Effects of Planar and In-Depth Motion on 3DTV QoE.

(2013)

Beitrag in Sammelwerk/Tagungsband

J. Paulus
G. Michelson
Marcus Barkowsky
J. Hornegger
B. Eskofier
M. Schmidt

Measurement of Individual Changes in the Performance of Human Stereoscopic Vision for Disparities at the Limits of the Zone of Comfortable Viewing.

pg. 310-317

(2013)

Zeitschriftenartikel

M. Urvoy
Marcus Barkowsky
P. Le Callet

How visual fatigue and discomfort impact 3D-TV quality of experience: a comprehensive review of technological, psychophysical, and psychological factors.

In: Annales des Télécommunications (vol. 68) , pg. 641-655

(2013)

DOI: 10.1007/s12243-013-0394-3

Abstract anzeigen

The quality of experience (QoE) of 3D contents is usually considered to be the combination of the perceived visual quality, the perceived depth quality, and lastly the visual fatigue and comfort. When either fatigue or discomfort are induced, studies tend to show that observers prefer to experience a 2D version of the contents. For this reason, providing a comfortable experience is a prerequisite for observers to actually consider the depth effect as a visualization improvement. In this paper, we propose a comprehensive review on visual fatigue and discomfort induced by the visualization of 3D stereoscopic contents, in the light of physiological and psychological processes enabling depth perception. First, we review the multitude of manifestations of visual fatigue and discomfort (near triad disorders, symptoms for discomfort), as well as means for detection and evaluation. We then discuss how, in 3D displays, ocular and cognitive conflicts with real world experience may cause fatigue and discomfort; these includes the accommodation-vergence conflict, the inadequacy between presented stimuli and observers depth of focus, and the cognitive integration of conflicting depth cues. We also discuss some limits for stereopsis that constrain our ability to perceive depth, and in particular the perception of planar and in-depth motion, the limited fusion range, and various stereopsis disorders. Finally, this paper discusses how the different aspects of fatigue and discomfort apply to 3D technologies and contents. We notably highlight the need for respecting a comfort zone and avoiding camera and rendering artifacts. We also discuss the influence of visual attention, exposure duration, and training. Conclusions provide guidance for best practices and future research.

Zeitschriftenartikel

W. Chen
J. Fournier
Marcus Barkowsky
P. Le Callet

Methodologies for Assessing 3D QoE: Standards and Explorative Studies.

In: ZTE Communications (vol. 11) , pg. 2-10

(2013)

Abstract anzeigen

Mastering quality of experience (QoE) is key to the widespread adoption of stereoscopic 3DTV (S-3DTV). However, assessing QoE of S-3DTV is not straightforward. Methods for determining observer experience need to be clearly defined and sufficiently robust. In this paper, we present state-of-the-art subjective QoE assessment for S-3DTV. We present conventional stan⁃ dardized ITU recommendations for evaluating picture quality and discuss new ITU activities in the area of S-3DTV assess⁃ ment. We also present and discuss explorative studies from the literature. We then introduce ways of using conventional quality assessment for S-3DTV QoE assessment. In discussing our pro⁃ posal, we mainly focus on QoE indicators and common features of subjective assessment. Multidimensional QoE indicators need to be used in S-3DTV to highlight advantages and reveal problems. In the second part of our proposal, we discuss the re⁃ quirements for adapting ITU-R BT.500, a conventional subjec⁃ tive QoE assessment method, ITU-R BT.500, for assessing QoE of S-3DTV are presented.

Beitrag in Sammelwerk/Tagungsband

M. Urvoy
Marcus Barkowsky
J. Li
P. Le Callet

Visual Comfort and Fatigue in Stereoscopy.

In:
C. Loscos
Y. Remion
L. Lucas

London, UK: ISTE Ltd, Wiley pg. 309-329

(2013)

Beitrag in Sammelwerk/Tagungsband

Marcus Barkowsky
N. Staelens
L. Janowski

Open collaboration on hybrid video quality models ‐ VQEG joint effort group hybrid.

pg. 476-481

(2013)

Beitrag in Sammelwerk/Tagungsband

M. Pinson
C. Schmidmer
L. Janowski
R. Pepion
Q. Huynh-Thu
P. Corriveau
A. Younkin
P. Le Callet
Marcus Barkowsky
W. Ingram

Subjective and objective evaluation of an audiovisual subjective dataset for research and development.

pg. 30-31

(2013)

Zeitschriftenartikel

K. Brunnström
I. Ananth
C. Hedberg
K. Wang
B. Andrén
Marcus Barkowsky

36.4: Comparison between Different Rating Scales for 3D TV.

In: SID Symposium Digest of Technical Papers (vol. 44) , pg. 509-512

(2013)

DOI: 10.1002/j.2168-0159.2013.tb06256.x

Abstract anzeigen

Stereoscopic 3D viewing experience has been studied quite intensively recently, but still the subjective test methods have not yet been settled. It has become clear that the 3D viewing experience cannot easily be described by just one scale. This paper describes a study where three different rating scales (Quality, Discomfort and Presence) are compared in a subjective test, combined with two viewing distances. The results shows that in a stereoscopic 3D video quality test targeting mainly coding distortions one scale such as video quality could be sufficient.

Zeitschriftenartikel

M. Pinson
Marcus Barkowsky
P. Le Callet

Selecting scenes for 2D and 3D subjective video quality tests.

In: EURASIP Journal on Image and Video Processing , pg. Article number: 50

(2013)

Abstract anzeigen

This paper presents recommended techniques for choosing video sequences for subjective experiments. Subjective video quality assessment is a well-understood field, yet scene selection is often driven by convenience or content availability. Three-dimensional testing is a newer field that requires new considerations for scene selection. The impact of experiment design on best practices for scene selection will also be considered. A semi-automatic selection process for content sets for subjective experiments will be proposed.

Beitrag in Sammelwerk/Tagungsband

J. Li
Marcus Barkowsky
P. Le Callet

Boosting Paired Comparison methodology in measuring visual discomfort of 3DTV: Performances of three different designs.

In:
SPIE - The International Society for Optical Engineering

(2013)

Beitrag in Sammelwerk/Tagungsband

Marcus Barkowsky
J. Li
T. Han
S. Youn
J. Ok
C. Lee
I. Vijai Ananth
K. Wang
K. Brunnström
P. Le Callet

Towards standardized 3DTV QoE assessment: Cross-lab study on display technology and viewing environment parameters.

In:
SPIE - The International Society for Optical Engineering

(2013)

Beitrag in Sammelwerk/Tagungsband

J. Li
Marcus Barkowsky
P. Le Callet

Subjective Assessment Methodology For Preference Of Experience In 3dtv.

Seoul, South Korea pg. 1

(2013)

Beitrag in Sammelwerk/Tagungsband

M. Leszczuk
L. Janowski
Marcus Barkowsky

Freely Available Large-scale Video Quality Assessment Database in Full-HD Resolution with H.264 Coding.

Atlanta, GA, USA vol. - pg. 1

(2013)

Beitrag in Sammelwerk/Tagungsband

Marcus Barkowsky
K. Brunnström
T. Ebrahimi
L. Karam
P. Lebreton
P. Le Callet
A. Perkis
A. Raake
M. Subedar
K. Wang
L. Xing
J. You

Subjective and Objective Visual Quality Assessment in the Context of Stereoscopic 3D-TV.

In:
Y. Zhao
C. Zhu
M. Tanimoto
L. Yu

Springer New York pg. 413-437

DOI: 10.1007/978-1-4419-9964-1_14

(2013)

Beitrag in Sammelwerk/Tagungsband

P. Lebreton
A. Raake
Marcus Barkowsky
P. Le Callet

Perceptual preference of S3D over 2D for HDTV in dependence of video quality and depth.

pg. 1-4

(2013)

Zeitschriftenartikel

P. Le Callet
Marcus Barkowsky

On viewing distance and visual quality assessment in the age of Ultra High Definition TV.

In: VQEG (Video Quality Expert Group) eLetter (vol. 1) , pg. 25-30

(2014)

Abstract anzeigen

The consumer video market is largely driven by the introduction of new formats(e.g.,new pixel resolution). Each time,the story remains the same: what is the optimal viewing distance? Ultra High Definition TV is not an exception. This simple question is of crucial importance when it comes to the issue of quality and the added value of a new technology. In this letter, werevisit the topic,starting from bestpractices and then raising open questions.

Zeitschriftenartikel

J. Li
Marcus Barkowsky
P. Le Callet

Validation of reliable 3DTV subjective assessment methodology ‐ Establishing a Ground Truth Database.

In: VQEG (Video Quality Expert Group) eLetter (vol. 1)

(2014)

Abstract anzeigen

Quality of Experience (QoE) in 3DTV is a multi-dimensional concept which includes image quality, depth quality, and visual comfort. How to measure this multi-dimensional concept is a challenging issue nowadays. In this letter, we introduce a Ground Truth database which is targeted for the standardization of subjective methodologies for QoE in 3DTV.

Beitrag in Sammelwerk/Tagungsband

P. Lebreton
A. Raake
Marcus Barkowsky
P. Le Callet

Evaluating complex scales through subjective ranking.

pg. 303-308

(2014)

Beitrag in Sammelwerk/Tagungsband

J. Li
Marcus Barkowsky
P. Le Callet

Assessing the Quality of Experience of 3DTV and Beyond: Tackling the Multidimensional Sensation.

In:
A. Kondoz
T. Dagiuklas

New York: Springer pg. 201-222

(2014)

Zeitschriftenartikel

J. Li
Marcus Barkowsky
P. Callet

Visual discomfort of stereoscopic 3D videos: Influence of 3D motion.

In: Displays (vol. 35) , pg. 49-57

(2014)

DOI: 10.1016/j.displa.2014.01.002

Abstract anzeigen

Visual discomfort is one of the most frequent complaints of the viewers while watching 3D images and videos. Large disparity and large amount of motion are two main causes of visual discomfort. To quantify this influence, three objectives are set in this paper. The first one is the comparative analysis on the influence of different types of motion, i.e., static stereoscopic image, planar motion and in-depth motion, on visual discomfort. The second one is the investigation on the influence factors for each motion type, for example, the disparity offset, the disparity amplitude and velocity. The third one is to propose an objective model for visual discomfort. Thirty-six synthetic stereoscopic video stimuli with different types of motion are used in this study. In the subjective test, an efficient paired comparison method called Adaptive Square Design (ASD) was used to reduce the number of comparisons for each observer and keep the results reliable. The experimental results showed that motion does not always induce more visual discomfort than static conditions. The in-depth motion generally induces more visual discomfort than the planar motion. The relative disparity between the foreground and the background, and the motion velocity are identified as main factors for visual discomfort. According to the subjective results, an objective model for comparing visual discomfort induced by different types of motion is proposed which shows high correlation with the subjective perception.

Beitrag in Sammelwerk/Tagungsband

K. Zhu
Marcus Barkowsky
M. Shen
P. Callet
D. Saupe

Optimizing feature pooling and prediction models of VQA algorithms.

pg. 541-545

(2014)

Beitrag in Sammelwerk/Tagungsband

P. Lebreton
A. Raake
Marcus Barkowsky
P. Le Callet

Measuring perceived depth in natural images and study of its relation with monocular and binocular depth cues.

In:
SPIE - The International Society for Optical Engineering

San Francisco, CA, USA pg. 1

(2014)

Beitrag in Sammelwerk/Tagungsband

J. Li
Y. Koudota
Marcus Barkowsky
H. Primon
P. Le Callet

Comparing Upscaling Algorithms From Hd To Ultra Hd By Evaluating Preference Of Experience.

(2014)

Zeitschriftenartikel

G. van Wallendael
N. Staelens
E. Masala
L. Janowski
K. Berger
Marcus Barkowsky

Dreamed about training, verifying and validating your QoE model on a million videos?.

In: VQEG (Video Quality Expert Group) eLetter (vol. 1) , pg. 19-29

(2014)

Beitrag in Sammelwerk/Tagungsband

P. Lebreton
Marcus Barkowsky
A. Raake
P. Le Callet

Chapter 20: 3D Video.

In:
S. Möller
A. Raake

Springer International Publishing pg. 299-313

(2014)

Zeitschriftenartikel

Marcus Barkowsky
I. Sedano
K. Brunnström
M. Leszczuk
N. Staelens

Hybrid video quality prediction: reviewing video quality measurement for widening application scope.

In: Multimedia Tools and Applications (vol. 74) , pg. 323-343

(2014)

DOI: 10.1007/s11042-014-1978-2

Abstract anzeigen

A tremendous number of objective video quality measurement algorithms have been developed during the last two decades. Most of them either measure a very limited aspect of the perceived video quality or they measure broad ranges of quality with limited prediction accuracy. This paper lists several perceptual artifacts that may be computationally measured in an isolated algorithm and some of the modeling approaches that have been proposed to predict the resulting quality from those algorithms. These algorithms usually have a very limited application scope but have been verified carefully. The paper continues with a review of some standardized and well-known video quality measurement algorithms that are meant for a wide range of applications, thus have a larger scope. Their individual artifacts prediction accuracy is usually lower but some of them were validated to perform sufficiently well for standardization. Several difficulties and shortcomings in developing a general purpose model with high prediction performance are identified such as a common objective quality scale or the behavior of individual indicators when confronted with stimuli that are out of their prediction scope. The paper concludes with a systematic framework approach to tackle the development of a hybrid video quality measurement in a joint research collaboration.

Zeitschriftenartikel

J. Li
A. Wang
Wang, J.
Marcus Barkowsky
P. Le Callet

Visual Discomfort Induced by Three-Dimensional Display Technology (in Chinese).

In: Laser and Optoelectronics Progress (vol. 52)

(2015)

Beitrag in Sammelwerk/Tagungsband

Y. Rai
Marcus Barkowsky
P. Le Callet

Does H.265 based peri and para-foveal quality flicker disrupt natural viewing patterns?.

pg. 133-136

(2015)

Beitrag in Sammelwerk/Tagungsband

P. Lebreton
A. Raake
Marcus Barkowsky
P. Le Callet

Open perceptual binocular and monocular descriptors for stereoscopic 3D images and video characterization.

pg. 1-6

(2015)

Beitrag in Sammelwerk/Tagungsband

G. van Wallendael
N. Staelens
E. Masala
Marcus Barkowsky

Full-HD HEVC-Encoded Video Quality Assessment Database.

(2015)

Beitrag in Sammelwerk/Tagungsband

A. Aldahdooh
Marcus Barkowsky
P. Le Callet

The impact of complexity in the rate-distortion optimization: A visualization tool.

pg. 45-48

(2015)

Zeitschriftenartikel

Marcus Barkowsky
E. Masala
G. van Wallendael
K. Brunnström
N. Staelens
P. Le Callet

Objective Video Quality Assessment ‐- Towards Large Scale Video Database Enhanced Model Development.

In: IEICE Transactions on Communications (vol. E-98b) , pg. 2-11

(2015)

Abstract anzeigen

The current development of video quality assessment algorithms suffers from the lack of available video sequences for training, verification and validation to determine and enhance the algorithm's application scope. The Joint Effort Group of the Video Quality Experts Group (VQEG-JEG) is currently driving efforts towards the creation of large scale, reproducible, and easy to use databases. These databases will contain bitstreams of recent video encoders (H.264, H.265), packet loss impairment patterns and impaired bitstreams, pre-parsed bitstream information into files in XML syntax, and well-known objective video quality measurement outputs. The database is continuously updated and enlarged using reproducible processing chains. Currently, more than 70,000 sequences are available for statistical analysis of video quality measurement algorithms. New research questions are posed as the database is designed to verify and validate models on a very large scale, testing and validating various scopes of applications, while subjective assessment has to be limited to a comparably small subset of the database. Special focus is given on the principles guiding the database development, and some results are given to illustrate the practical usefulness of such a database with respect to the detailed new research questions.

Beitrag in Sammelwerk/Tagungsband

M. Shahid
J. Panasiuk
G. van Wallendael
Marcus Barkowsky
B. Lovstrom

Predicting full-reference video quality measures using HEVC bitstream-based no-reference features.

pg. 1-2

(2015)

Beitrag in Sammelwerk/Tagungsband

K. Berger
Y. Koudota
Marcus Barkowsky
P. Le Callet

Subjective Quality Assessment Comparing UHD and HD Resolution in HEVC Transmission Chain.

(2015)

Beitrag in Sammelwerk/Tagungsband

P. Lebreton
A. Raake
Marcus Barkowsky

Studying user agreement on aesthetic appeal ratings and its relation with technical knowledge.

DOI: 10.1109/QoMEX.2016.7498934

(2016)

Beitrag in Sammelwerk/Tagungsband

A. Aldahdooh
E. Masala
O. Janssens
G. Wallendael
Marcus Barkowsky

Comparing simple video quality measures for loss-impaired video sequences on a large-scale database.

pg. 1-6

(2016)

Beitrag in Sammelwerk/Tagungsband

P. Lebreton
A. Raake
Marcus Barkowsky

Evaluation of aesthetic appeal with regard of user\textquoterights knowledge.

(2016)

Beitrag in Sammelwerk/Tagungsband

Y. Rai
A. Aldahdooh
S. Ling
Marcus Barkowsky
P. Callet

Effect of content features on short-term video quality in the visual periphery.

(2016)

Beitrag in Sammelwerk/Tagungsband

A. Aldahdooh
Marcus Barkowsky
P. Callet

Spatio-temporal error concealment technique for high order multiple description coding schemes including subjective assessment.

pg. 1-6

(2016)

Beitrag in Sammelwerk/Tagungsband

A. Aldahdooh
E. Masala
G. Wallendael
Marcus Barkowsky

Comparing temporal behavior of fast objective video quality measures on a large-scale database.

(2016)

Beitrag in Sammelwerk/Tagungsband

Y. Rai
Marcus Barkowsky
P. Le Callet

Role of spatio-temporal distortions in the visual periphery in disrupting natural attention deployment.

DOI: 10.2352/ISSN.2470-1173.2016.16.HVEI-117

(2016)

Zeitschriftenartikel

A. Aldahdooh
E. Masala
G. Wallendael
Marcus Barkowsky

Framework for reproducible objective video quality research with case study on PSNR implementations.

In: Digital Signal Processing (vol. 77) , pg. 195-206

(2018)

DOI: 10.1016/j.dsp.2017.09.013

Abstract anzeigen

Reproducibility is an important and recurrent issue in objective video quality research because the presented algorithms are complex, depend on specific implementations in software packages or their parameters need to be trained on a particular, sometimes unpublished, dataset. Textual descriptions often lack the required detail and even for the simple Peak Signal to Noise Ratio (PSNR) several mutations exist for images and videos, in particular considering the choice of the peak value and the temporal pooling. This work presents results achieved through the analysis of objective video quality measures evaluated on a reproducible large scale database containing about 60,000 HEVC coded video sequences. We focus on PSNR, one of the most widespread measures, considering its two most common definitions. The sometimes largely different results achieved by applying the two definitions highlight the importance of the strict reproducibility of the research in video quality evaluation in particular. Reproducibility is also often a question of computational power and PSNR is a computationally inexpensive algorithm running faster than realtime. Complex algorithms cannot be reasonably developed and evaluated on the abovementioned 160 hours of video sequences. Therefore, techniques to select subsets of coding parameters are then introduced. Results show that an accurate selection can preserve the variety of the results seen on the large database but with much lower complexity. Finally, note that our SoftwareX accompanying paper presents the software framework which allows the full reproducibility of all the research results presented here, as well as how the same framework can be used to produce derived work for other measures or indexes proposed by other researchers which we strongly encourage for integration in our open framework.

Zeitschriftenartikel

A. Aldahdooh
E. Masala
O. Janssens
G. Wallendael
Marcus Barkowsky
P. Callet
G. van Wallendael
P. Lambert

Improved Performance Measures for Video Quality Assessment Algorithms Using Training and Validation Sets.

In: IEEE Transactions on Multimedia (vol. 74) , pg. 32-41

(2018)

Abstract anzeigen

Due to the three-dimensional spatiotemporal regularities of natural videos and small-scale video quality databases, effective objective video quality assessment (VQA) metrics are difficult to obtain but highly desirable. In this paper, we propose a general-purpose no-reference VQA framework that is based on weakly supervised learning with convolutional neural network (CNN) and resampling strategy. First, an eight-layer CNN is trained by weakly supervised learning to construct the relationship between the deformations of the three dimensional discrete cosine transform of video blocks and corresponding weak labels judged by a full-reference (FR) VQA metric. Thus, the CNN obtains the quality assessment capacity converted from the FR-VQA metric, and the effective features of the distorted videos can be extracted through the trained network. Then, we map the frequency histogram calculated from the quality score vectors predicted by the trained network onto the perceptual quality. Specially, to improve the performance of the mapping function, we transfer the frequency histogram of the distorted images and videos to resample the training set. The experiments are carried out on several widely used video quality assessment databases. The experimental results demonstrate that the proposed method is on a par with some state-of-the-art VQA metrics and has promising robustness.

Zeitschriftenartikel

Andreas Gegenfurtner
Armin Eichinger
Richard Latzel
Marc-Philipp Dietrich
Marcus Barkowsky
Alexandra Glufke
Angelika Stadler
Wolfgang Stern

Mobiles Eye-Tracking in den angewandten Wissenschaften.

In: Bavarian Journal of Applied Sciences (vol. 4) , pg. 370-395

(2018)

DOI: 10.25929/bjas.v4i1.54

Abstract anzeigen

Mobiles Eye-Tracking ist als Forschungsmethode beliebter denn je und gewinnt in unterschiedlichen Feldern der angewandten Wissenschaften mehr und mehr an Bedeutung. Dieser Beitrag diskutiert, wie die Aufzeichnung und Analyse von Blickbewegungen in der Mobilität, im Usability Engineering, den Sportwissenschaften, der Augmented Reality/Mixed Reality/Virtual Reality und der Medizin bzw. medizinischen Weiterbildung eingesetzt wird. Der Beitrag gliedert sich dabei in drei Teile: in einem ersten Teil werden Grundzüge des Eye-Trackings erläutert; in einem zweiten Teil wird der Einsatz mobilen Eye-Trackings in ausgewählten Feldern der angewandten Wissenschaften veranschaulicht; und in einem abschließenden dritten Teil werden Potentiale und Risiken sowie zukünftige Forschungslinien skizziert, um die Anwendung mobilen Eye-Trackings als digitale Forschungsmethode weiter zu etablieren.

Zeitschriftenartikel

K. Brunnström
Marcus Barkowsky

Statistical quality of experience analysis: on planning the sample size and statistical significance testing.

In: Journal of Electronic Imaging (vol. 27) , pg. 053013

(2018)

DOI: 10.1117/1.JEI.27.5.053013

Abstract anzeigen

This paper analyzes how an experimenter can balance errors in subjective video quality tests between the statistical power of finding an effect if it is there and not claiming that an effect is there if the effect is not there, i.e., balancing Type I and Type II errors. The risk of committing Type I errors increases with the number of comparisons that are performed in statistical tests. We will show that when controlling for this and at the same time keeping the power of the experiment at a reasonably high level, it is unlikely that the number of test subjects that are normally used and recommended by the International Telecommunication Union (ITU), i.e., 15 is sufficient but the number used by the Video Quality Experts Group (VQEG), i.e., 24 is more likely to be sufficient. Examples will also be given for the influence of Type I error on the statistical significance of comparing objective metrics by correlation. We also present a comparison between parametric and nonparametric statistics. The comparison targets the question whether we would reach different conclusions on the statistical difference between the video quality ratings of different video clips in a subjective test, based on the comparison between the student T-test and the Mann‐Whitney U-test. We found that there was hardly a difference when few comparisons are compensated for, i.e., then almost the same conclusions are reached. When the number of comparisons is increased, then larger and larger differences between the two methods are revealed. In these cases, the parametric T-test gives clearly more significant cases, than the nonparametric test, which makes it more important to investigate whether the assumptions are met for performing a certain test.

Zeitschriftenartikel

J. Li
Wang, J.
Marcus Barkowsky
P. Callet

Exploring the effects of subjective methodology on assessing visual discomfort in immersive multimedia.

In: Electronic Imaging, Human Vision and Electronic Imaging

(2018)

DOI: 10.2352/ISSN.2470-1173.2018.14.HVEI-527

Abstract anzeigen

Visual discomfort is an important factor that influences viewing experience in immersive multimedia, for example, 3DTV and VR. With the added value of depth, the novel perceptual experience, visual discomfort is not an easy task for observers to evaluate. In this study, we investigate how the subjective methodology affects the test results in 3DTV condition. Two subjective visual discomfort experiments were conducted. One used the Pair Comparison (PC) method and the other used the Absolute-Category Rating (ACR) method. The results demonstrated that PC method had more powerful discriminability. For a difficult perceptualrelated tasks, such as visual discomfort in our study, PC was more easy to understand and conduct for the observers which led to reliable results. It also showed some very important but usually ignored conclusions on the subjective experiment, i.e., for measuring the perceived visual discomfort, the observer\textquoterights judgment behavior might be affected by the test methodology.

Zeitschriftenartikel

A. Aldahdooh
Marcus Barkowsky
P. Le Callet

Proof-of-concept: role of generic content characteristics in optimizing video encoders.

In: Multimedia Tools and Applications (vol. 77) , pg. 16069-16097

(2018)

DOI: 10.1007/s11042-017-5180-1

Abstract anzeigen

The influence of content characteristics on the efficiency of redundancy and irrelevance reduction in video coding is well known. Each new standard in video coding includes additional coding tools that potentially increase the complexity of the encoding process in order to gain further rate-distortion efficiency. In order to be versatile, encoder implementations often neglect the content dependency or they optimize the encoding complexity on a local scale, i.e. on a single frame or on the coding unit level without being aware of the global content type. In this contribution, an analysis is presented which coding tool settings of the recent High Efficiency Video Coding (HEVC) standard are most efficient for a given content type when balancing rate-distortion against computational complexity measured in encoding time. The content type is algorithmically determined, leading to a framework for rate-distortion-complexity based encoder parameter decision for any given video sequence. The implementability is demonstrated using a set of 35 Ultra-HD (UHD) sequences. The performance results and evaluations show that the encoding parameters may be predicted to optimize the video coding. For instance, predicting motion search range achieves complexity reduction of 36% on average when HEVC reference HM is used at a cost of bitrate (2%). When another HEVC coding standard software, x265, is used to predict the coding unit (CU) size, there is a reduction of 20% in bitrate and of 8% in distortion but there is a reduction of 6% in execution time.

Zeitschriftenartikel

A. Aldahdooh
E. Masala
G. Wallendael
Marcus Barkowsky

Reproducible research framework for objective video quality measures using a large-scale database approach.

In: SoftwareX (vol. 8) , pg. 64-68

(2018)

DOI: 10.1016/j.softx.2017.09.004

Abstract anzeigen

This work presents a framework to facilitate reproducibility of research in video quality evaluation. Its initial version is built around the JEG-Hybrid database of HEVC coded video sequences. The framework is modular, organized in the form of pipelined activities, which range from the tools needed to generate the whole database from reference signals up to the analysis of the video quality measures already present in the database. Researchers can re-run, modify and extend any module, starting from any point in the pipeline, while always achieving perfect reproducibility of the results. The modularity of the structure allows to work on subsets of the database since for some analysis this might be too computationally intensive. To this purpose, the framework also includes a software module to compute interesting subsets, in terms of coding conditions, of the whole database. An example shows how the framework can be used to investigate how the small differences in the definition of the widespread PSNR metric can yield very different results, discussed in more details in our accompanying research paper Aldahdooh etÂ al. (0000). This further underlines the importance of reproducibility to allow comparing different research work with high confidence. To the best of our knowledge, this framework is the first attempt to bring exact reproducibility end-to-end in the context of video quality evaluation research.

Beitrag in Sammelwerk/Tagungsband

Katharina Heydn
Marc-Philipp Dietrich
Marcus Barkowsky
Götz Winterfeldt
S. Mammen
A. Nüchter

The Golden Bullet: A Comparative Study for Target Acquisition, Pointing and Shooting.

In:
Institute of Electrical and Electronics Engineers Inc.

DOI: 10.1109/VS-Games.2019.8864589

(2019)

Zeitschriftenartikel

T. Mizdos
Marcus Barkowsky
M. Uhrina
P. Pocta

Linking Bitstream Information to QoE: A Study on Still Images Using HEVC Intra Coding.

In: Advances in Electrical and Electronic Engineering (AEEE) (vol. 17) , pg. 436-445

(2019)

DOI: 10.15598/aeee.v17i4.3625

Abstract anzeigen

The coding tools used in image and video encoders aim at high perceptual quality for low bitrates. Analyzing the results of the encoders in terms of quantization parameter, image partitioning, prediction modes or residuals may provide important insight into the link between those tools and the human perception. As a first step, this contribution analyzes the possibility to transcode reference images of three well-known image databases, i.e. IRCCyN/IVC, LIVE and TID2013, from their original, older formats to HEVC; thus creating a homogeneous database of 327 HEVC encoded images accompanied with bitstream parameters and values obtained from objective and subjective assessments. Secondly, it analyzes some of the HEVC intra coding parameters regarding their influence on the image quality by using machine learning, namely Support Vector Machine - Regression.

Beitrag in Sammelwerk/Tagungsband

L. Tiotsop
E. Masala
A. Aldahdooh
G. Wallendael
Marcus Barkowsky

Computing Quality-of-Experience Ranges for Video Quality Estimation.

In:
Institute of Electrical and Electronics Engineers Inc.

DOI: 10.1109/QoMEX.2019.8743303

(2019)

Zeitschriftenartikel

A. Aldahdooh
E. Masala
G. Wallendael
P. Lambert
Marcus Barkowsky

Improving relevant subjective testing for validation: Comparing machine learning algorithms for finding similarities in VQA datasets using objective measures.

In: Signal Processing: Image Communication (vol. 74) , pg. 32-41

(2019)

DOI: 10.1016/j.image.2019.01.004

Abstract anzeigen

Subjective quality assessment is a necessary activity to validate objective measures or to assess the performance of innovative video processing technologies. However, designing and performing comprehensive tests requires expertise and a large effort especially for the execution part. In this work we propose a methodology that, given a set of processed video sequences prepared by video quality experts, attempts to reduce the number of subjective tests by selecting a subset with minimum size which is expected to yield the same conclusions of the larger set. To this aim, we combine information coming from different types of objective quality metrics with clustering and machine learning algorithms that perform the actual selection, therefore reducing the required subjective assessment effort while trying to preserve the variety of content and conditions needed to ensure the validity of the conclusions. Experiments are conducted on one of the largest publicly available subjectively annotated video sequence dataset. As performance criterion, we chose the validation criteria for video quality measurement algorithms established by the International Telecommunication Union.

Beitrag in Sammelwerk/Tagungsband

L. Tiotsop
T. Mizdos
M. Uhrina
P. Pocta
Marcus Barkowsky
E. Masala

Predicting Single Observers Votes from Objective Measures using Neural Networks.

(2020)

Beitrag in Sammelwerk/Tagungsband

L. Tiotsop
T. Mizdos
E. Masala
Marcus Barkowsky
P. Pocta

How to Train No Reference Video Quality Measures for New Coding Standards using Existing Annotated Datasets?.

In:
Institute of Electrical and Electronics Engineers Inc.

IEEE pg. 1-6

DOI: 10.1109/MMSP53017.2021.9733456

(2021)

Zeitschriftenartikel

T. Mizdos
Marcus Barkowsky
M. Uhrina
P. Pocta

How to reuse existing annotated image quality datasets to enlarge available training data with new distortion types.

In: Multimedia Tools and Applications (vol. 80) , pg. 28137–28159

(2021)

DOI: 10.1007/s11042-021-10679-5

Abstract anzeigen

There is a continuing demand for objective measures that predict perceived media quality. Researchers are developing new methods for mapping technical parameters of digital media to the perceived quality. It is quite common to use machine learning algorithms for these purposes especially deep learning algorithms, which need large amounts of data for training. In this paper, we aim towards getting more training data with recent types of distortions. Instead of doing expensive subjective experiments, we evaluate the reuse of previously published, well-known image datasets with subjective annotation. In this contribution, the procedure of mapping Mean Opinion Scores (MOS) from an already published subjectively annotated dataset with older codecs to new codecs is presented. In particular, we map from Joint Photographic Experts Group (JPEG) distortions to newer High Efficiency Video Coding (HEVC) distortions. We have used values of three different objective methods as a connection between these two different distortion types. In order to investigate the significance of our approach, subjective verification tests were designed and conducted. The design goals led to two types of experiments, i.e. Pair Comparison (PC) test and Absolute Category Rating (ACR) test, in which 40 participants provided their opinion. Results of the subjective experiments indicate that it may be possible to use information gained from older datasets to describe the perceived quality of more recent compression algorithms.

Zeitschriftenartikel

L. Tiotsop
T. Mizdos
M. Uhrina
Marcus Barkowsky
P. Pocta
E. Masala

Modeling and estimating the subjects’ diversity of opinions in video quality assessment: a neural network based approach.

In: Multimedia Tools and Applications (vol. 80) , pg. 3469-3487

(2021)

DOI: 10.1007/s11042-020-09704-w

Abstract anzeigen

Subjective experiments are considered the most reliable way to assess the perceived visual quality. However, observers’ opinions are characterized by large diversity: in fact, even the same observer is often not able to exactly repeat his first opinion when rating again a given stimulus. This makes the Mean Opinion Score (MOS) alone, in many cases, not sufficient to get accurate information about the perceived visual quality. To this aim, it is important to have a measure characterizing to what extent the observed or predicted MOS value is reliable and stable. For instance, the Standard deviation of the Opinions of the Subjects (SOS) could be considered as a measure of reliability when evaluating the quality subjectively. However, we are not aware of the existence of models or algorithms that allow to objectively predict how much diversity would be observed in subjects’ opinions in terms of SOS. In this work we observe, on the basis of a statistical analysis made on several subjective experiments, that the disagreement between the quality as measured by means of different objective video quality metrics (VQMs) can provide information on the diversity of the observers’ ratings on a given processed video sequence (PVS). In light of this observation we: i) propose and validate a model for the SOS observed in a subjective experiment; ii) design and train Neural Networks (NNs) that predict the average diversity that would be observed among the subjects’ ratings for a PVS starting from a set of VQMs values computed on such a PVS; iii) give insights into how the same NN based approach can be used to identify potential anomalies in the data collected in subjective experiments.

Zeitschriftenartikel

L. Tiotsop
F. Agboma
G. van Wallendael
A. Aldahdooh
S. Bosse
L. Janowski
Marcus Barkowsky
E. Masala

On the Link Between Subjective Score Prediction and Disagreement of Video Quality Metrics.

In: IEEE Access (vol. 9) , pg. 152923-152937

(2021)

DOI: 10.1109/ACCESS.2021.3127395

Abstract anzeigen

Several video quality metrics (VQMs) have been proposed in many publications to predict how humans perceive video quality. It is common to observe significant disagreements amongst the quality predictions of these VQMs for the same video sequence. Following an extensive literature search, we found no publicised work that has investigated if such disagreements convey useful information on the accuracy of VQMs. Herein, a measure for quantifying the disagreement between VQMs is proposed. A small-scale subjective study is carried out to assess the effectiveness of our proposal. In particular, the proposed disagreement measure is shown to be extremely effective in determining whether the quality of any given processed video sequence (PVS) can be accurately predicted by the VQMs. This type of information is particularly useful for identifying video sequences that are likely to degrade the end-user’s quality of experience (QoE). Our proposal is also useful in selecting the most effective PVSs to be employed in a subjective test. We show that the proposed disagreement measure can be effectively predicted from bitstream features. This establishes a link between the capability to accurately assess the quality of a PVS and the way it is encoded. In addition, an analysis is conducted to compare the performances of some well-known and widely used open-source metrics and two proprietary metrics. The two proprietary metrics are used by a large media company for enhancing its delivery pipeline. The outcome of this comparison highlights the suitability of the open-source VQM, Video Multi-method Assessment Fusion (VMAF), as a good benchmark quality measure for both the industrial and academic environments.

Zeitschriftenartikel

L. Tiotsop
T. Mizdos
Marcus Barkowsky
P. Pocta
A. Servetti
E. Masala

Mimicking Individual Media Quality Perception with Neural Network based Artificial Observers.

In: ACM Transactions on Multimedia Computing, Communications, and Applications (vol. 18) , pg. 1-25

(2022)

DOI: 10.1145/3464393

Abstract anzeigen

The media quality assessment research community has traditionally been focusing on developing objective algorithms to predict the result of a typical subjective experiment in terms of Mean Opinion Score (MOS) value. However, the MOS, being a single value, is insufficient to model the complexity and diversity of human opinions encountered in an actual subjective experiment. In this work we propose a complementary approach for objective media quality assessment that attempts to more closely model what happens in a subjective experiment in terms of single observers and, at the same time, we perform a qualitative analysis of the proposed approach while highlighting its suitability. More precisely, we propose to model, using neural networks (NNs), the way single observers perceive media quality. Once trained, these NNs, one for each observer, are expected to mimic the corresponding observer in terms of quality perception. Then, similarly to a subjective experiment, such NNs can be used to simulate the users’ single opinions, which can be later aggregated by means of different statistical indicators such as average, standard deviation, quantiles, etc. Unlike previous approaches that consider subjective experiments as a black box providing reliable ground truth data for training, the proposed approach is able to consider human factors by analyzing and weighting individual observers. Such a model may therefore implicitly account for users’ expectations and tendencies, that have been shown in many studies to significantly correlate with visual quality perception. Furthermore, our proposal also introduces and investigates an index measuring how much inconsistency there would be if an observer was asked to rate many times the same stimulus. Simulation experiments conducted on several datasets demonstrate that the proposed approach can be effectively implemented in practice and thus yielding a more complete objective assessment of end users’ quality of experience.

Zeitschriftenartikel

T. Mizdos
Marcus Barkowsky
P. Pocta
M. Uhrina

30 Years of Video Coding Evolution - What Can We Learn from it in Terms of QoE?.

In: Advances in Electrical and Electronic Engineering (AEEE) (vol. 20)

(2022)

DOI: 10.15598/aeee.v20i3.3998

Abstract anzeigen

From the beginnings of ITU-T H.261 to H.265 (HEVC), each new video coding standard has aimed at halving the bitrate at the same perceptual quality by redundancy and irrelevancy reduction. Each improvement has been explained by comparably small changes in the video coding toolset. This contribution aims at starting the Quality of Experience (QoE) analysis of the accumulated improvements over the last thirty years. Based on an overview of the changes in the coding tools, we analyze the changes in the quantized residual information. Visual comparison and statistical measures are performed and some interpretations are provided towards explaining how irrelevancy reduction may have led to such a huge reduction in bitrate. The interpretation of the results in terms of QoE paves the way towards an understanding of the coding tools in terms of visual quality. It may help in understanding how the irrelevancy reduction has been improved over the decades. Understanding how the differences of the residuals relate to known or yet unknown properties of the human visual system, may enable a closer collaboration between perception research and video compression research.

Beitrag in Sammelwerk/Tagungsband

L. Tiotsop
A. Servetti
Marcus Barkowsky
E. Masala

Regularized Maximum Likelihood Estimation of the Subjective Quality from Noisy Individual Ratings.

Institute of Electrical and Electronics Engineers, Inc. pg. 1-4

DOI: 10.1109/QoMEX55416.2022.9900903

(2022)

Beitrag in Sammelwerk/Tagungsband

P. Majer
L. Tiotsop
Marcus Barkowsky

Training the DNN of a Single Observer by Conducting Individualized Subjective Experiments.

IEEE pg. 103-106

DOI: 10.1109/QoMEX58391.2023.10178608

(2023)

Zeitschriftenartikel

L. Tiotsop
A. Servetti
Marcus Barkowsky
P. Pocta
T. Mizdos
G. van Wallendael
E. Masala

Predicting individual quality ratings of compressed images through deep CNNs-based artificial observers.

In: Signal Processing: Image Communication (vol. 112) , pg. 116917

(2023)

DOI: 10.1016/j.image.2022.116917

Abstract anzeigen

Unlike traditional objective approaches aimed at MOS prediction, subjective experiments provide individual opinion scores that allow, for instance, to estimate the distribution of users’ opinion scores. Unfortunately, the current literature is lacking objective quality assessment approaches that simulate the process of a subjective test. Therefore, this work focuses on modeling an individual subject through a deep CNN that, once trained, is expected to mimic the subject in terms of quality perception; for this reason, we call it “Artificial Intelligence-based Observer” (AIO). Several AIOs, modeling subjects with different characteristics, can be derived and used to simulate the process of a subjective test, thus yielding a more complete objective quality assessment. However, the training of the AIOs is hindered by two major issues: (i) the lack of training sets containing a large number of individual opinion scores; (ii) the noisy nature of individual opinion scores used as ground truth. To overcome these issues, we motivate a two-step learning approach. During the first learning step, the architecture of the well-known ResNet50 is appropriately modified and its initial weights are updated using a large scale synthetically annotated dataset of JPEG compressed images created for quality assessment purpose. This yields a new deep CNN called JPEGResNet50 that can accurately evaluate the quality of JPEG compressed images. The second learning step, conducted on a subjectively annotated dataset, refines the generic perceptual quality features already learned by the JPEGResNet50 to derive the AIO of each subject. Extensive computational experiments show the potential and effectiveness of our approach.

Zeitschriftenartikel

L. Tiotsop
A. Servetti
P. Pocta
G. van Wallendael
Marcus Barkowsky
E. Masala

Multiple Image Distortion DNN Modeling Individual Subject Quality Assessment.

In: ACM Transactions on Multimedia Computing, Communications, and Applications (vol. 20) , pg. 1-27

(2024)

DOI: 10.1145/3664198

Abstract anzeigen

A recent research direction is focused on training Deep Neural Networks (DNNs) to replicate individual subject assessments of media quality. These DNNs are referred to as Artificial Intelligence-based Observers (AIOs). An AIO is designed to simulate, in real-time, the quality ratings of a specific individual, enabling an automatic quality assessment that accounts for subjects characteristics and preferences. Training AIOs is a promising but challenging research area due to the greater noise in individual raw opinion scores compared to the Mean Opinion Score. Effective learning from noisy labels necessitates the training of complex models on large-scale datasets. Unfortunately, this is challenging for AIOs as the media quality assessment community lacks extensive datasets that include individual opinion scores. To address the complexity of the task, we first created a dataset comprising two million samples, with synthetic labels derived from human annotation. We then trained a customized network for image quality assessment, named Multi-Distortion ResNet50 (MDResNet50), on this dataset. The weights of the MDResNet50 were subsequently utilized to initialize the learning process of each AIO, thereby avoiding the need to train a complex model from scratch on a small-scale dataset with raw individual opinion scores. Computational experiments show that our approach significantly advances the state-of-the-art in the AIO research. In particular: (i) we demonstrate through a simulation the ability of AIOs to mimic two well-known behavioral characteristics of a subject, i.e., bias and inconsistency, when scoring the media quality; (ii) we train and release DNN-based AIOs that, compared to the state-of-the-art, exhibit a higher performance with a statistical significance in assessing multiple image distortions; (iii) we train AIOs that more accurately mimic the sensitivity of real subjects to noise and color saturation and also better predict the opinion score distribution compared to the state-of-the-art AIOs.

Zeitschriftenartikel

L. Tiotsop
A. Servetti
Marcus Barkowsky
E. Masala

Modeling Subject Scoring Behaviors in Subjective Experiments Based on a Discrete Quality Scale.

In: IEEE Transactions on Multimedia (vol. 26) , pg. 8742-8757

(2024)

DOI: 10.1109/TMM.2024.3382483

Abstract anzeigen

Several approaches have been proposed to estimate quality in subjective experiments while highlighting peculiar subject behaviors. However, there is some room for improvement in existing approaches, both in terms of robustness to noise and the ability to accurately indicate several peculiar subject behaviors in subjective experiments. This work advances the state-of-the-art in three main directions: i) A new approach to estimate the subjective quality from noisy ratings is proposed and is shown to be more robust to noise than are four state-of-the-art approaches; ii) a novel subject scoring model is proposed that makes it possible to highlight several peculiar behaviors typically observed in subjective experiments; and iii) our proposed probabilistic subject scoring model results from the proof of a theorem, whereas in previous approaches a probabilistic scoring model is assumed a priori. This represents an important first step toward models supported by a stronger theoretical foundation. Numerical experiments conducted on several datasets highlight the effectiveness of our proposal.